1. Field
This disclosure relates generally to computers, and, more particularly, to clustered client-server architecture infrastructure.
2. Background
Most businesses use multiple networked servers, in some cases, many, many servers networked into clusters to handle their IT infrastructure needs. However, it is well known that applications/programs rarely use the full capabilities of existing servers on which they run, particularly for typical server deployments in a clustering environment, e.g., data center servers, leaving many servers with underutilized, excess capacity. Such server capacity may be underutilized for various reasons, for example, to allow for performance spikes or due to the possibility of interference between applications/programs.
Because of the way some programs are written so as to be able to communicate with the “outside world” (i.e., other computers on the network, be it a small network within the company or the internet) while not knowing where they will be installed, if one needs to add an application/program in a clustering environment it is extremely difficult to use that underutilized, excess server capacity along side the existing (i.e., “legacy”) applications/programs because, since the applications do not know where they will be installed, port assignment is done dynamically at installation using a technique called “dynamic discovery” that allows the new application to identify its location and advertise its presence on a network.
Dynamic discovery operates such that, when the application is initially installed or run on a server, the application will “announce” to the network “I am here” and provide a zero value for the port value indicating where “here” is. The operating system, which maintains a table of all used and available ports for each server (denoted by a “Host Name”), will then intercept that initial announce and assign a free port in place of the zero value to denote where “here” is, and thereafter, that is the port through which the application/program will be reachable from the other programs. Various programs may be used to implement the service for dynamic discovery including, for example, in the Linux environment, Apache ZooKeeper, Consul, Etcd, Eureka, etc.
However, there is a class of applications, interchangeably referred to herein as “ensemble applications” and “ensemble programs” that involve multiple instances of the same program being deployed and run on multiple servers. One example application within that class is Apache Cassandra, which is an open source distributed database management system designed to handle large amounts of data across many servers involving clusters spanning multiple datacenters. In general, ensemble applications specify or require that all instances use the same port number(s) irrespective of the host servers on which they are running. That is because the applications typically use proprietary network connections among the various instances.
Unfortunately, a current trend is to use “lightweight containers” for applications in clustered server environments. This creates a technological impediment that prevents, or makes very difficult, the use of any existing excess server capacity for applications of this class. That is because each lightweight container must be isolated from any other lightweight containers (hereafter interchangeably referred to as “legacy components” or “legacy containers”) on the same server(s), i.e., they cannot use any of the same ports. This in enforced through the dynamic discovery process such that each lightweight container has its ports independently dynamically assigned without regard to any other lightweight container.
As a result, presently, the only way to ensure independence when adding a new lightweight container to an existing server would be to add a new hostname for that lightweight container on that existing Host (if possible). However, adding a new hostname requires significant additional detailed work, additional IP addresses, it requires new security for that Host, and additional complexity that must be supported, etc. Moreover, adding an ensemble application into such a container adds an additional layer of problems due to the need to have all instances use the same ports as noted above.
Dynamic discovery is antithetical to that requirement because (i) it cannot guarantee the same port assignments to all instances, and (ii) the required ports for the instances may already be allocated to other legacy applications.
Thus, there is presently a significant technological problem inhibiting the ability to make use of excess server capacity by adding ensemble applications that run in lightweight containers onto one or more servers that already contain legacy components.